Imaging apparatus and control method for tracking a subject based on an image signal

ABSTRACT

An imaging apparatus including an electric charge accumulation unit configured to accumulate electric charge in accordance with an incident light quantity; a tracking unit configured to perform tracking; and an exposure control unit configured to perform exposure control, wherein the electric charge accumulation unit performs a first accumulation, and then performs a second accumulation after the first accumulation, wherein the tracking unit performs the tracking based on an image signal obtained through the first accumulation, and wherein the exposure control unit computes a control value for the exposure control based on at least the image signal obtained through the second accumulation of the image signal obtained through the first accumulation and an image signal obtained through the second accumulation.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an imaging apparatus capable oftracking a subject.

2. Description of the Related Art

There is conventionally an imaging apparatus equipped with a so-calledsubject tracking function which recognizes the position of a mainsubject in an imaging frame and automatically tracks its position. Byusing this subject tracking function, it is possible to continuouslyperform automatic focusing control (AF control) and exposure controlwith respect to a moving main subject.

As an example of an imaging apparatus equipped with such a subjecttracking function, Japanese Patent Application Laid-Open No. 7-110429discusses an imaging apparatus such as a single-lens reflex camera whichhas a light metering unit for light metering separate from an imagingelement and which tracks a subject based on light metering data outputfrom the light metering unit. However, in the case where subjecttracking is performed based on light metering data output from a lightmetering unit as in the case of the imaging apparatus discussed inJapanese Patent Application Laid-Open No. 7-110429, the following issuesarise.

In an ordinary single-lens reflex camera like the imaging apparatusdiscussed in Japanese Patent Application Laid-Open No. 7-110429, theposition of a quick return mirror provided in the camera is moved toswitch between a state in which the incident light from the subject isguided to an imaging element and a state in which it is guided to aneyepiece (optical finder).

When the incident light from the subject is guided to the eyepiece, theincident light from the subject is also guided to a light metering unit,making is possible to perform light metering on the subject. In otherwords, in the state in which the incident light from the subject isguided to the imaging element, light metering cannot be performed at thelight metering unit.

More specifically, when the state in which the incident light from thesubject is guided to the imaging element and the state in which it isguided to the light metering unit, are successively switched with ashort period, as in the case of continuous shooting, the accumulationtime of light metering at the light metering unit is limited. Inparticular, when the light metering data output from the light meteringunit is used for subject tracking, it is necessary to complete thesubject tracking computation before AF control or exposure control isperformed, with the result that the accumulation time is further limitedthan the period in which the incident light from the subject is guidedto the light metering unit.

As a result, the accumulation time of the light metering unit is shorterthan the flicker period, which results in an increase in the influenceof the flicker, making it unlikely to obtain an accurate light meteringvalue. Here, the flicker will be described. When light metering isperformed under a fluorescent lamp or the like, there occurs a so-calledflicker phenomenon, in which the brightness of the illumination lightperiodically varies due to the influence of an AC power frequency. In anarea where the power frequency is 50 Hz, the flicker period isapproximately 10 ms; in an area where it is 60 Hz, the flicker period isapproximately 8.3 ms. When light metering is performed in such anenvironment, if the accumulation time of the light metering unit is notan integral multiple of the flicker period, a light metering valuevaries depending on a portion of the phase of the flicker period withwhich the accumulation period overlaps. Thus, it is difficult to performexposure control in a stable manner.

If, in view of this, the accumulation time of the light metering unit islengthened in order to reduce the influence of the flicker, therequisite time until completing the subject tracking computation is alsolengthened, resulting in reduction in a shooting speed at the time ofcontinuous shooting.

SUMMARY OF THE INVENTION

According to an aspect of the present invention, an imaging apparatusincludes: an electric charge accumulation unit configured to accumulateelectric charge in accordance with an incident light quantity; atracking unit configured to perform tracking; and an exposure controlunit configured to perform exposure control, wherein the electric chargeaccumulation unit performs a first accumulation, and then performs asecond accumulation after the first accumulation, wherein the trackingunit performs the tracking based on an image signal obtained through thefirst accumulation, and wherein the exposure control unit computes acontrol value for the exposure control based on at least the imagesignal obtained through the second accumulation of the image signalobtained through the first accumulation and an image signal obtainedthrough the second accumulation.

Further features and aspects of the present invention will becomeapparent from the following detailed description of exemplaryembodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate exemplary embodiments, features,and aspects of the invention and, together with the description, serveto explain the principles of the invention.

FIG. 1 is a diagram schematically illustrating the construction of animaging apparatus according to an exemplary embodiment of the presentinvention.

FIG. 2 illustrates an operational sequence during shooting in anexemplary embodiment of the present invention.

FIGS. 3A, 3B, 3C, 3D, 3E, and 3F are diagrams schematically illustratinga light metering operation in an exemplary embodiment of the presentinvention.

FIG. 4 is a diagram illustrating a difference due to a pixel additionnumber of a read image in a light metering sensor according to anexemplary embodiment of the present invention.

FIG. 5 is a diagram illustrating a light metering computation in anexemplary embodiment of the present invention.

FIGS. 6A, 6B, 6C, and 6D are diagrams illustrating an AF controlperformed following a movement of a tracking object.

FIGS. 7A, 7B, 7C, and 7D are diagrams illustrating an AF controlperformed following a movement of a tracking object in a case where aface detection result is utilized in an exemplary embodiment of thepresent invention.

DESCRIPTION OF THE EMBODIMENTS

Various exemplary embodiments, features, and aspects of the inventionwill be described in detail below with reference to the drawings.

In the following, a favorable exemplary embodiment of the presentinvention will be illustrated in detail with reference to theaccompanying diagrams. FIG. 1 is a diagram schematically illustratingthe construction of an imaging apparatus according to the exemplaryembodiment of the present invention; the imaging apparatus is composedof a camera main body 100 and a lens unit 200 detachable from the cameramain body 100.

A microcomputer central processing unit (CPU) 101 controls each portionof the camera main body 100. An imaging device 102 such as acharge-coupled device (CCD) or a complementary metal oxide semiconductor(CMOS) including an infrared cut filter, a low-pass filter or the likephoto-electrically converts light having entered the imaging apparatusvia a photographing lens of the lens unit 200 to output an image signal.

A mechanical shutter 103 is operated to be switched between a stateshielding the imaging device 102 from the light having entered via thephotographing lens and a retracted state in which it guides the lighthaving entered via the photographing lens to the imaging device 102.

A half mirror 104 is movable to a position where it guides the lighthaving entered via the photographing lens to the imaging device 102 (themirror-up state: second state) and a position where it guides the lightto a light metering sensor 106 (mirror-down state: first state). Inother words, the half mirror 104 switches the optical path of the lighthaving entered via the photographing lens between the state guiding thelight to the imaging device 102 and the state guiding the light to thelight metering sensor 106. When the half mirror is at the position whereit guides the light to the light metering sensor 106, the light havingentered via the photographing lens is directed to a focusing plate 105to form an image.

The light metering sensor 106 includes an imaging device such as a CCD,which photo-electrically converts a subject image to accumulate electriccharge to output an image signal. The present exemplary embodimentdescribed below employs, as the light metering sensor 106, a CCD havingM×N pixels arranged in a matrix-like manner.

A pentagonal prism 107 guides the light having entered via thephotographing lens and reflected by the half mirror 104, to the lightmetering sensor 106 and an optical finder (not illustrated). Memory 108such as random access memory (RAM), and read only memory (ROM) isconnected to the CPU 101. An auto focus (AF) mirror 110 reflects a partof the light having entered via the photographing lens and passedthrough the half mirror 104, and guides it to an AF sensor in a focusdetection circuit 109. The AF sensor has a plurality of AF regions in animaging frame, making it possible to obtain the defocusing amount ineach AF region.

The CPU 101 transmits AF control information based on the output fromthe focus detection circuit 109 to a lens CPU 201 (hereinafter referredto as the LCPU) configured to control each portion of the lens unit 200.The LCPU 201 drives the photographing lens in the lens unit 200 based onthe received AF control information to perform automatic focusadjustment control (hereinafter referred to as the AF control).

A CPU 111 (hereinafter referred to as the ICPU) performs imageprocessing and computation processing on an image signal output from thelight metering sensor 106 after electric charge is accumulated. The ICPUperforms subject tracking, light metering, face detection, etc.illustrated below based on the image signal output from the lightmetering sensor 106. The result of the light metering computation by theICPU 111 is transmitted to the CPU 101, and the CPU 101 sets shutterspeed, stop value, etc. based on the result of the light meteringcomputation, and performs exposure control. Memory 112 such as RAM, andROM is connected to the ICPU 111.

While in the present exemplary embodiment the CPU for performing imageprocessing and computation processing on the image signal output fromthe light metering sensor 106 is provided separately from the CPU forcontrolling each portion of the camera main body 100, it is alsopossible for a single CPU to perform both processing.

Further, the lens unit 200 may be contained in the camera main body 100or the CPU 101 may drive the photographing lens to perform AF control.

Next, the operational sequence at the time of shooting in the presentexemplary embodiment will be illustrated with reference to FIG. 2. FIG.2 illustrates the operational sequence in continuous shooting from themirror-down state prior to the shooting of the Nth frame (N≧2) to themirror-down state prior to the shooting of the (N+2)th frame.

First, for the shooting of the Nth frame, the light metering sensor 106performs accumulation a1 (first accumulation) in the mirror-down state.This accumulation a1 is performed in order to obtain an image signal tobe used for subject tracking. The accumulation time is set by the ICPU111 such that the continuous shooting speed at the time of continuousshooting is not reduced and that the result of the computation for thesubject tracking can be reflected in the AF control. Thus, when theshooting speed at the time of continuous shooting is higher than apredetermined value, the accumulation time is set to a time shorter thanthe flicker period. However, when the accumulation time is too short, asufficient accumulation is not effected, and the computation for subjecttracking cannot be performed, so that the accumulation time is set to atime as required for obtaining an image signal enabling the computationof subject tracking.

Although not illustrated in FIG. 2, accumulation by an AF sensor in thefocus detection circuit 109 for performing the AF control is alsoperformed in parallel with the accumulation a1. Defocusing amountcomputation etc. of each AF region is performed based on the result ofthe accumulation by the AF sensor. After that, AF control is performedbased on the defocusing amount computation result, tracking computationresult illustrated below, etc.

In this way, subject tracking is performed based on the image signalobtained through accumulation a1, so that it is possible to perform anAF control in which the tracking result is reflected without reducing ashooting speed at the time of continuous shooting.

When the accumulation a1 is completed, a reading process Ra1, in whichthe image signals accumulated are read out, is started. Further, inparallel with the reading Ra1, accumulation b1 (second accumulation) isperformed. This accumulation b1 is processing for performing a lightmetering computation in which the influence of the flicker is reduced;its setting is made such that the light metering computation can becompleted until the start of the exposure control. In a case where it isdesirable not to perform the accumulation b1 in parallel with thereading Ra1, the accumulation b1 may be started after the reading Ra1has been completed. This also applies to the following reading andaccumulation.

In this way, the accumulation b1 is performed subsequent to theaccumulation a1, and light metering computation is performed based onthe image signal obtained through the accumulation a1 and the imagesignal obtained through the accumulation b1, so that it is possible toperform an exposure reducing the influence of the flicker.

As illustrated in FIGS. 3A and 3B, when the accumulation a1 is shorterthan the flicker period, the luminance value based on the accumulatedimage signals varies depending upon a portion of the phase of theflicker period with which the accumulation a1 overlaps. In view of this,as illustrated in FIGS. 3C and 3D, the accumulation b1 is performedsubsequent to the accumulation a1, and the light metering computation isperformed based on the image signals accumulated in each of them. As aresult, as compared with the case in which the light meteringcomputation is performed based solely on the image signals accumulatedin the accumulation a1, the total accumulation time of the image signalsused for light metering computation is lengthened. Thus, the influenceof the flicker is reduced as compared with the case in which the lightmetering computation is performed based solely on the image signalsaccumulated in the accumulation a1, making it possible to perform a moreaccurate exposure control. Further, the phase of the overlapping flickerperiod is made different between the accumulation a1 and accumulationb1, so that the fluctuation in the average value of the flicker sourcelight quantity during the accumulation periods combining theaccumulation a1 and accumulation b1 is reduced, thus stabilizing thelight metering computation result and making it possible to perform amore accurate exposure control.

If the sum total of the accumulation time of the accumulation a1 and theaccumulation tie of the accumulation b1 is set to be substantially equalto an integral multiple of the flicker period, it is possible to furtherreduce the influence of the flicker.

Next, when the reading Rat is completed, subject tracking computation isstarted. It is necessary for the subject tracking computation to becompleted by the time when the AF control is started, and the ICPU 111sets the accumulation time of the accumulation a1 such that the trackingcomputation is completed by the time the AF control is started.

And, when the accumulation b1 is completed, reading Rb1, in which theaccumulated image signals are read out, is started. Further,accumulation c1 (third accumulation) is performed in parallel with thereading Rb1. This accumulation c1 is processing for obtaining an imagesignal to be used for the face detection of the subject. However, theface detection computation requires more time as compared with thetracking computation and light metering computation, so that if thedetection result is to be reflected in the AF control or the like forthe shooting performed immediately after the completion of theaccumulation c1, a shooting speed is reduced at the time of continuousshooting. If the shooting speed is not to be reduced at the time ofcontinuous shooting, the face detection computation cannot be completedin time for the AF control or the like for the shooting performedimmediately after the completion of the accumulation c1. In view ofthis, the result of the face detection computation using the imagesignals obtained through the accumulation c1 is used for the AF controlnot for the shooting performed immediately after the completion of theaccumulation c1 but for the next shooting.

Reading Rc1, in which the image signals accumulated in the accumulationc1 are read can be executed even in the mirror-up state, so that theaccumulation time of the accumulation c1 is set by the ICPU 111 suchthat the series of processing from the start of the accumulation a1 tothe completion of the accumulation c1 are completed in the mirror-downstate. Before the reading Rc1, in which the image signals accumulated inthe accumulation c1 are read, is completed, a mirror-up operation isstarted. Thus, the face detection computation is performed after themirror-up operation has been started. Similarly, after this processing,accumulations a2, b2, and c2 and accumulations a3, b3, and c3 areperformed in the mirror-down state before the shooting of the (N+1) thframe and the (N+2) th frame is carried out.

As illustrated above, in the present exemplary embodiment, theaccumulations a1, b1, and c1 are performed in the mirror-down state;however, it is also possible to perform an intermittent accumulation ineach accumulation in order to reduce the influence of the flicker invarious processing.

As shown in FIGS. 3A and 3B, when the accumulation a1 is shorter thanthe flicker period, the luminance value based on the accumulated imagesignals varies depending on a portion of the phase of the flicker periodwith which the accumulation period overlaps.

In view of this, as illustrated in FIG. 3E, a plurality of accumulationsare performed intermittently, with each accumulation being performed fora time obtained by dividing the accumulation time when accumulation isperformed collectively, by the number of accumulations. In other words,the accumulation time illustrated in FIG. 3A is equal to the sum totalof the accumulation time of the plurality of accumulations illustratedin FIG. 3E. And, the image signal obtained through the intermittentaccumulation is used as the image signal obtained through theaccumulation a1.

When intermittent accumulation is performed, the phase of the flickerperiod overlapping with the accumulation time is dispersed, so that itis possible to reduce the influence of the flicker as compared with thecase where normal accumulation is performed.

In the above intermittent accumulations, instead of reading the imagesignal each time accumulation is completed, it is desirable to integratethe image signals accumulated in each accumulation and read the imagesignals after all the accumulations have been completed. By thisprocessing, the reading of the image signal after each accumulation canbe omitted, making it possible to reduce the processing load.

Such intermittent accumulation may also be executed in at least one ofthe accumulations a1, b1, and c1, or may be switched between whetherintermittent accumulation is to be performed or not according to eachaccumulation time. For example, intermittent accumulation may beperformed in the case where the accumulation time is shorter than theflicker period and where the influence of the flicker is expected to begreat.

Further, as illustrated in FIG. 3F, when intermittent accumulation isperformed in both the accumulation a1 and accumulation b1, the phase ofthe flicker period overlapping with the accumulation period is furtherdispersed as compared with the case of FIGS. 3C and 3D, thus making itpossible to achieve a further reduction of the influence of the flicker.

Next, an image signal reading method according to the present exemplaryembodiment will be illustrated with reference to FIG. 4. In the presentexemplary embodiment, a stripe type CCD is used as the light meteringsensor 106. In the stripe type CCD, the pixels of red (R), green (G),and blue (B) are arranged in a stripe-like manner as illustrated in FIG.4; in such a stripe type CCD, it is possible to perform pixel additionin an analog manner in the vertical direction.

When, for example, pixel addition is performed with four pixels, thesize of the image based on the read image signal is reduced to ¼ in thevertical direction. At this time, due to the pixel addition, the outputlevel of the image signal is approximately four times the level when nopixel addition is performed (when all the pixels are read), so that itis possible to obtain an image of a brightness equivalent to that of theimage based on the image signal read without performing pixel additionin an accumulation time reduced to approximately ¼. Further, the imagesignal read amount is also reduced as compared with that when no pixeladdition is performed, so that it is possible to shorten the requisitetime for image signal reading.

Further, in the reading Ra1, Rb1, by performing pixel addition readingas illustrated above, it is possible to obtain an image signal enablingsubject tracking computation in a short accumulation time also intracking a dark subject. Through the pixel addition reading, a reductionin accumulation time and an increase in the influence of the flicker areto be expected; however, when combined with the above-illustratedintermittent accumulation, it is possible to suppress the influence ofthe flicker.

When pixel addition is performed, the image resolution deteriorates.Therefore, detection accuracy is reduced in a case where subject facedetection is performed through recognition of features such as the casewhere it is determined whether a human face exists in an image, throughmatching with a previously prepared face pattern. Thus, when performingsubject face detection by such a method, it is desirable to perform allpixel reading without performing pixel addition in the reading Rc1, inwhich the image signals accumulated in the accumulation c1 for obtainingan image signal for subject face detection, are read.

Next, light metering computation according to the present exemplaryembodiment will be illustrated with reference to FIG. 5. As illustratedin FIG. 5, the image based on the image signal obtained throughaccumulation at the light metering sensor 106 is divided into I (in thevertical direction)×J (in the horizontal direction) light meteringregions. Here, each light metering region includes m (in the verticaldirection)×n (in the horizontal direction) pixels; I×m=M, and J×n=N;thus, the light metering sensor 106 as a whole has M×N pixels.

And, in each light metering region, the respective average outputs Ra,Ga, and Ba of the R, G, and B pixels are calculated to calculate theluminance value Yij of each light metering region from the calculatedRa, Ga, and Ba. The luminance value Yij is obtained, for example, by thefollowing equation (1):Yij=A×Ra+B×Ga+C×Ba  (1)By substituting appropriate values into the mixing ratios A, B, and C ofthe R, G, and B pixels, the luminance Yij is obtained.

Next, weighted average computation is performed, in which a weight isassigned to the Yij of each light metering region according to the lightmetering mode such as evaluation light metering or spot light meteringto obtain the luminance value of the entire light metering region. Theabove light metering computation is performed on the image signalobtained through the accumulation a1 and the image signal obtainedthrough the accumulation b1 to acquire the respective luminance valuesY1 and Y2.

Here, assuming that the accumulation time of the accumulation a1 is T1and that the accumulation time of the accumulation b1 is T2, the finalluminance value Y to be used for the exposure control can be expressedby the following equation (2).Y=(Y1/T1+Y2/T2)×T1/2  (2)The above equation (2) is used to obtain the average value of theluminance values when the accumulations a1 and b1 are both performed inthe accumulation time T1; however, the formula for obtaining the finalluminance value is not restricted to equation (2). For example, it mayalso be an equation for obtaining the average value of the luminancevalues when the accumulations a1 and b1 are both performed in theaccumulation time T2. In obtaining the final luminance value in thisway, the weighted average of the luminance value Y1 and the luminancevalue Y2 is obtained. Further, it may also be an equation for obtainingthe average value of the luminance values when the accumulations a1 andb1 are both performed in a predetermined accumulation time Ts.

Further, equation (3) may also be employed, in which the accumulationtime of the accumulation a1 and the accumulation time of theaccumulation b1 are equalized and in which the accumulation time of eachof them is reduced to half the flicker period, thus simplifying theweighted average computation.Y=(Y1+Y2)/2  (3)By setting each accumulation time such that the total accumulation time,which is the sum total of the accumulation time of the accumulation a1and the accumulation time of the accumulation b1, is the same as theflicker period, it is possible to further reduce the influence of theflicker.

The accumulation time of the accumulations a1, b1, and c1 may be set topreviously determined times, or may be set by the ICPU 111 based on thepreceding light metering result. In the case where the setting is madeby the ICPU 111 based on the result of the preceding light meteringcomputation, the setting is made, for example, such that theaccumulation time is shorter when the final luminance value Y is highthan when the final luminance value Y obtained from the preceding lightmetering computation result is low.

Next, the subject tracking computation will be illustrated. In thetracking computation in the present exemplary embodiment, comparison ismade, for example, between the image obtained through the accumulationa1 and the image obtained through the accumulation a2, and computation(pattern matching) is performed to determine where the tracking objecthas moved, based on the distribution of the luminance pattern and colorpattern of the tracking object satisfying a predetermined condition.More specifically, tracking of an object satisfying a predeterminedcondition is performed in the shooting frame. And, AF control isperformed while changing the AF region where focusing is effectedfollowing the movement of the tracking object. Alternatively, exposurecontrol is performed while changing the weighting with respect to eachlight metering region following the movement of the tracking object.

FIG. 6 is a diagram illustrating AF control following the movement of atracking object. In the initial state of continuous shooting, when themain subject exists at the position of FIG. 6A within the shootingframe, an AF frame 7 at the position where the main subject exits is setas the frame indicating the AF region for focusing to perform AFcontrol. In FIGS. 6A through 6D, the AF frame set as the frameindicating the AF region for focusing is indicated by a frame thickerthan the other AF frames.

After AF control has been performed to effect focusing at a regioncorresponding to the AF frame 7, an accumulation is performed to obtainan image signal to be used for subject tracking. The ICPU 111 sets apredetermined region using the AF frame 7 as a reference as the trackingregion in the obtained image. In the tracking region, the matching ofluminance pattern and color pattern is performed in the trackingcomputation. While it is also possible to read the image signal used forthe subject tracking computation by performing pixel addition, in thecase of the image illustrated in FIGS. 6A through 6D, all pixels areread.

After this, suppose the position where the main subject exists has movedto the position of FIG. 6B, and accumulation for obtaining the imagesignal for the next subject tracking is performed. To determine whatposition the tracking object has moved in the obtained image, the ICPU111 sets a predetermined region, using the tracking region set based onthe AF frame 7 as a reference, as a search region, which is a regionwhere the tracking object is searched for. The search region includesthe tracking region, and is set to be larger than the tracking region.

Within the search region of the image obtained this time, the ICPU 111determines whether there exists a region matching the tracking region ofthe image obtained the last time in terms of luminance pattern and colorpattern. When the search region is set large, tracking is possible evenif the position of the tracking object has greatly changed; however, thelarger the search region, the longer the requisite time fordetermination, so that the search region is set by taking into accountthe tracking accuracy and the time that can be utilized for the trackingcomputation.

When it is determined that there exists within the search region aregion matching the tracking region, the AF frame existing within theregion is set as the AF frame indicating the AF region where the nextfocusing is effected. In FIG. 6B, the AF frame 8 corresponds to thatframe. Then, shooting is performed after AF control has been performedso as to effect focusing in a region corresponding to the AF frame 8.

When it is determined that there exists within the search region aregion matching the tracking region, it can be determined that thereexists a matching region if there exists a region whose degree ofmatching with the tracking region is not less than a predeterminedvalue.

Further, the ICPU 111 newly sets a region matching the tracking regionof the image obtained previously, in the image obtained currently, asthe tracking region.

After this, suppose the position where the main subject exists has movedto the position as illustrated in FIG. 6C, and, in this state,accumulation for obtaining the image signal for the next subjecttracking has been performed. To determine to what position the trackingobject has moved within the image thus obtained this time, the ICPU 111sets the search region using the position of the tracking region as areference. More specifically, since the tracking region newly set thelast time is a region centering on the AF frame 8, the search region isaccordingly changed from the preceding one.

Similarly, in FIG. 6D, in which the position of the main subject hasmoved from the position in FIG. 6C, the search region is changed toperform the search of the tracking object. After this processing,tracking computation is performed prior to each shooting at the time ofcontinuous shooting, so that it is always possible to perform shootingfocusing on the main subject taken as the tracking object in the initialstate.

In each tracking computation at the time of continuous shooting, if thetracking computation results in failure, for example, in the case wherethe tracking computation has resulted in failure in the state of FIG. 6Cof FIGS. 6A through 6D, in carrying out the tracking computation in thestate of FIG. 6D, the tracking computation can be performed based on thetracking region of FIG. 6B. More specifically, the search of a regionmatching the tracking region of the image illustrated in FIG. 6B isperformed. In this case, the position of the subject is expected tochange more than in the case where the tracking computation is performedbetween successive images, so that the search may be performed bysetting the search region wider than normal.

Alternatively, when the tracking computation results in failure in eachtracking computation at the time of continuous shooting, the trackingobject may be re-set again. When the accumulation time of each imageused for the tracking computation of the subject is shorter than theflicker period, brightness may vary due to the influence of the flicker.In such a case, image comparison may be performed after processing formaking a brightness level of the images substantially the same iscarried out.

Further, it is also possible to use the face detection computationresult for the above-described subject tracking computation; by usingthe face detection result, it is possible to perform the subjecttracking more accurately. In the following, the subject trackingcomputation using the face detection result will be illustrated withreference to FIG. 7. FIG. 7 is a diagram illustrating an AF controlfollowing the movement of the tracking object in the case where the facedetection result is used. Regarding the technique for detecting a humanface, any one of the well-known methods may be adopted; for example, itis possible to adopt a method in which light metering data istransformed into hue and saturation. By preparing and analyzing atwo-dimensional histogram thereof, a face region can be determined.Alternatively, it is possible to adopt a method in which a facecandidate region corresponding to human face features is extracted and aface region is determined from the feature amount in the region.

In the case where the main subject exists at the position of FIG. 7A inthe initial state of continuous shooting, AF control is performed so asto focus on the AF region corresponding to the AF frame 7 situated wherethe main subject exists. In FIGS. 7A through 7D, the AF frame set as theframe indicating the AF region where focusing is effected is indicatedby a thicker frame than the other AF frames. In this state, a regionother than the face of the main subject may be set as the AF regionwhere focusing is effected.

And, in the state in which focusing has been effected on the regioncorresponding to the AF frame 7, accumulation is performed to obtain theimage signal to be used for subject tracking and accumulation to obtainthe image signal to be used for the subject face detection. Then, facedetection computation is performed based on the obtained image signal toset the subject face region detected as the tracking region in the imageto be used for the subject tracking.

After this, suppose, when the position where the main subject exists hasbeen changed to the position of FIG. 7B, accumulation is performed toobtain the image signal to be used for the next subject tracking andaccumulation to obtain the image signal to be used for the subject facedetection. To determine to what position the tracking object has movedin the image thus obtained this time, a search region is set using theposition of the tracking region in the image obtained the last time as areference. More specifically, in the image illustrated in FIG. 7A, aregion near the AF frame 2 is used as the subject face region accordingto the face detection computation, and this face region is set as thetracking region, so that the search region is set to a predeterminedregion using this face detection region as a reference.

Then, within the search region of the image obtained this time, a searchis made as to whether there exists a region matching the luminancepattern and color pattern of the tracking region of the image obtainedthe last time.

In the case where there exists within the search region a regionmatching the tracking region, the AF frame existing in that region isset as the AF frame indicating the AF region for the next focusing. InFIG. 7B, the AF frame 3 corresponds to that frame. Then, shooting isperformed after AF control is performed so as to focus on a regioncorresponding to the AF frame 3.

In this case, the face detection computation using the image signalsaccumulated when the subject exists at the position of FIG. 7B cannot beperformed in time for the AF control, so that the result of the facedetection computation is used for the tracking computation in the nextAF control.

Further, face detection computation is performed using the image signalsaccumulated when the subject exists at the position of FIG. 7B. Thedetected subject face region is newly set as the tracking region in theimage to be used for subject tracking. In this way, by setting thetracking region of each tracking computation based on the result of theface detection computation, it is possible to accurately track the faceregion of the tracking object.

When the region matching the preceding tracking region in the image usedfor the subject tracking computation obtained this time and the detectedsubject face region are substantially the same, a region matching thepreceding tracking region may be set as the new tracking region.

After this, suppose, when the position where the main subject exists hasbeen moved to the position of FIG. 7C, the accumulation is performed toobtain the image signal to be used for the next subject tracking and theaccumulation is performed to obtain the image signal to be used for thesubject face detection.

To determine to what position the tracking object has moved in the imagethus obtained this time, a search region is set using the position ofthe tracking region in the image obtained last time. More specifically,the tracking region of the preceding image is a region which centers onthe AF frame 3 where the subject face region exists in the state of FIG.7B, so that the search region is accordingly changed from the precedingone. In the case where a region matching the tracking region existswithin the search region, the AF frame existing within that region isset as the AF frame indicating the AF region for the next focusing. InFIG. 7C, the AF frame 4 corresponds to that frame. Then, shooting isperformed after AF control has been performed to focus on the regioncorresponding to the AF frame 4.

Also at the position illustrated in FIG. 7D, to which the position ofthe main subject has been moved from the position in FIG. 7C, thetracking object is searched for, while the face region is detected bythe face detection computation using the image signals accumulated whenthe subject exists at the position of FIG. 7C, the face region being setas the tracking region for the search.

Also after this processing, tracking computation is performed using theface detection result, prior to each shooting at the time of continuousshooting, so that it is always possible to perform shooting whilefocusing on the main subject used as the tracking object in the initialstate, in particular, the face region of the main subject.

In FIG. 7, the AF control is performed to focus on the AF regioncorresponding to the face region of the detected subject. However, alsoin the region below the face region of the subject, the subject distanceis regarded substantially equal to that in the face region, so that inthe case, for example, where no AF region corresponding to the faceregion of the subject detected exists, focusing may be effected on theAF region corresponding to the region below the face region of thedetected subject.

As illustrated above, accumulation operation by the light meteringsensor 106 is performed a plurality of times prior to each shooting atthe time of continuous shooting, and subject tracking is performed basedon the image signals obtained through the first accumulation operation,so that it is possible to perform subject tracking without reducing ashooting speed at the time of continuous shooting. Further, exposurecontrol is performed based on image signals obtained throughaccumulation operation performed a plurality of times, so that when theaccumulation time of a single accumulation operation is shorter than theflicker period, it is possible to perform exposure control reducing theinfluence of the flicker.

Further, the face detection result based on the image signals obtainedthrough accumulation by the light metering sensor 106 in the precedingmirror-down state is used in the tracking computation for the shootingthis time. Therefore, it is possible to perform subject tracking inwhich the detection result of the subject detection is reflected,without reducing a continuous shooting speed.

While in the present exemplary embodiment described above, the luminancepattern and color pattern serving as a reference in tracking computationare updated each time an image to be used for subject tracking isobtained, it is also possible to perform tracking computation whilemaintaining the luminance pattern and color pattern first determined asthe tracking reference.

Further, while in the present exemplary embodiment the accumulation forlight metering computation is performed a plurality of times, such asaccumulation a1 and accumulation b1, it is not necessary to perform theaccumulation for light metering computation a plurality of times in thecase where continuous shooting is not performed or in the case wheresufficient accumulation time can be secured by a single accumulation,for example, before the shooting of the first frame in continuousshooting. However, the accumulation for light metering computation maybe performed a plurality of times even in the case where continuousshooting is not performed or in the case where sufficient accumulationtime can be secured by a single accumulation, for example, before theshooting of the first frame in continuous shooting.

Further, in the case where continuous shooting is performed, when thecontinuous shooting speed is higher than a predetermined value, theaccumulation for light metering computation is performed a plurality oftimes. However, there is no need to perform the accumulation for lightmetering computation a plurality of times when the continuous shootingspeed is not higher than the predetermined value.

Further, even in the case where the accumulation is performed aplurality of times, such as accumulation a1 and accumulation b1, thereis no need to employ the image signals obtained through the accumulationa1 for the light metering computation if sufficient accumulation time issecured in the accumulation b1.

In addition, to further reduce the influence of the flicker, theaccumulation for light metering computation may be performed in a largerquantity than in the exemplary embodiment illustrated above.

Further, while in the present exemplary embodiment the detection of theface region of a human subject is performed through face detectioncomputation, it is also possible to perform subject detection, using asubject region satisfying other predetermined conditions as thedetection object. For example, the user may previously set a subjectregion as the detection object, and the set subject region may bedetected in the subject detection computation thereafter.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications, equivalent structures, and functions.

This application claims priority from Japanese Patent Applications No.2011-029970 filed Feb. 15, 2011 and No. 2011-029996 filed Feb. 15, 2011,which are hereby incorporated by reference herein in their entirety.

What is claimed is:
 1. An imaging apparatus which is capable ofcontinuous shooting comprising: an electric charge accumulation unitconfigured to accumulate electric charge in accordance with an incidentlight quantity; an imaging unit which is different from the electriccharge accumulation unit; an optical path changing unit configured toswitch an optical path of light entering via a photographing lensbetween a first state in which the light is guided to the electriccharge accumulation unit, and a second state in which the light isguided to the imaging unit, wherein the optical path changing unitplaces the optical path in the second state when shooting is to beperformed by the imaging unit; a tracking unit configured to performtracking; and an exposure control unit configured to perform exposurecontrol, wherein the optical path changing unit repeatedly switches theoptical path between the first state and the second state in continuousshooting, wherein the electric charge accumulation unit performs a firstaccumulation and then performs a second accumulation after the firstaccumulation from when the optical path is changed from the second stateto the first state for a next shooting in continuous shooting until theoptical path is changed from the first state to the second state for thenext shooting, wherein the tracking unit performs the tracking for thenext shooting based on an image signal obtained through the firstaccumulation, and wherein the exposure control unit computes exposurecondition for the exposure control of the next shooting based on a thirdluminance value, which is computed based on a first luminance valuecomputed based on the image signal obtained through the firstaccumulation and a second luminance value computed based on the imagesignal obtained through the second accumulation, wherein when theshooting speed at the time of continuous shooting is higher than apredetermined value, an accumulation time of the first accumulation isset to a time shorter than a flicker period.
 2. The imaging apparatusaccording to claim 1, wherein the third luminance value is a weightedaverage value of the first luminance value and the second luminancevalue.
 3. The imaging apparatus according to claim 1, furthercomprising: a focus adjustment unit configured to perform focusadjustment based on the tracking result of the tracking unit, whereinthe tracking unit completes a tracking computation before the focusadjustment unit performs the focus adjustment for the next shooting. 4.The imaging apparatus according to claim 3, wherein the focus adjustmentunit performs the focus adjustment when the optical path is in thesecond state for the next shooting.
 5. The imaging apparatus accordingto claim 3, wherein the electric charge accumulation unit performs thefirst accumulation in an accumulation time enabling the tracking unit tocomplete the tracking computation before the focus adjustment unitperforms the focus adjustment.
 6. The imaging apparatus according toclaim 1, wherein the electric charge accumulation unit performs thefirst accumulation and the second accumulation such that the sum totalof the accumulation time of the first accumulation and the accumulationtime of the second accumulation is substantially equal to an integralmultiple of a flicker period.
 7. The imaging apparatus according toclaim 1, wherein accumulation is performed a plurality of times in anintermittent manner in at least one of the first accumulation and thesecond accumulation, and wherein the electric charge accumulation unitoutputs the image signal after all the intermittent accumulations havebeen completed.
 8. The imaging apparatus according to claim 1, whereinthe tracking unit performs the tracking of an object satisfying apredetermined condition.
 9. The imaging apparatus according to claim 1,wherein the electric charge accumulation unit outputs an image signalthat has undergone pixel addition through the first accumulation. 10.The imaging apparatus according to claim 1, wherein the electric chargeaccumulation unit outputs an image signal that has undergone pixeladdition through the second accumulation.
 11. The imaging apparatusaccording to claim 1, further comprising: a subject detection unitconfigured to detect a subject region, wherein the electric chargeaccumulation unit performs a third accumulation after the secondaccumulation, and wherein the subject detection unit detects the subjectregion based on an image signal obtained through the third accumulation.12. The imaging apparatus according to claim 11, wherein the trackingunit performs the tracking based on the image signal obtained throughthe first accumulation and the detection result of the subject detectionunit.
 13. The imaging apparatus according to claim 11, wherein theelectric charge accumulation unit performs the first accumulation, thesecond accumulation, and the third accumulation from when the opticalpath is changed from the second state to the first state for the nextshooting until the optical path is changed from the first state to thesecond state for the next shooting.
 14. The imaging apparatus accordingto claim 10, wherein the electric charge accumulation unit outputs animage signal that has not undergone pixel addition through the thirdaccumulation.
 15. A method of controlling an imaging apparatus which iscapable of continuous shooting including: accumulating electric chargein accordance with an incident light quantity using an electric chargeaccumulation unit; switching, using an optical path changing unit, anoptical path of light entering via a photographing lens between a firststate in which the light is guided to the electric charge accumulationunit, and a second state in which the light is guided to an imaging unitwhich is different from the electric charge accumulation unit, whereinthe optical path changing unit places the optical path in the is beperformed by the imaging unit; performing tracking using a trackingunit; performing exposure control using an exposure control unit;switching repeatedly the optical path between the first state and thesecond state in continuous shooting using the optical path changingunit; performing a first accumulation and then performs a secondaccumulation after the first accumulation from when the optical path ischanged from the second state to the first state for a next shooting incontinuous shooting until the optical path is changed from the firststate to the second state for the next shooting; performing the trackingby the tracking unit for the next shooting based on an image signalobtained through the first accumulation; and computing exposurecondition for the exposure control of the next shooting based on a thirdluminance value, which is computed based on a first luminance valuecomputed based on the image signal obtained through the firstaccumulation and a second luminance value computed based on the imagesignal obtained through the second accumulation, wherein when theshooting speed at the time of continuous shooting is higher than apredetermined value, an accumulation time of the first accumulation isset to a time shorter than a flicker period.
 16. The imaging apparatusaccording to claim 1, wherein the exposure control unit computes ashutter speed based on the image signal obtained through the secondaccumulation.
 17. The imaging apparatus according to claim 1, whereinthe exposure control unit computes a stop value based on the imagesignal obtained through the second accumulation.